{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c54878f8-00e5-4caf-af20-427b3a040842",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copyright 2023 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f00c49d7-82de-48e0-b807-0a3d06f04082",
   "metadata": {},
   "source": [
    "# Evaluate Classification\n",
    "| | |\n",
    "|----------|-------------|\n",
    "| Author(s)   | Renato Leite (renatoleite@), Egon Soares (egon@) |\n",
    "| Last updated | 09/05/2023 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7762fc3-b707-4980-a8a5-9e5d2037a8d5",
   "metadata": {},
   "source": [
    "## Per Class"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb2a566-2e9d-49ea-b05e-71c671ae05d0",
   "metadata": {},
   "source": [
    "- Dataset used for this sample\n",
    "<cite>\n",
    "  <a href=\"https://www.aclweb.org/anthology/D18-1404\">CARER: Contextualized Affect Representations for Emotion Recognition</a> by Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687-3697, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.\n",
    "</cite>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bb63c4dc-7fb2-42b4-b8c7-a3a7eed85d55",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from https://github.com/dair-ai/emotion_dataset - modified to binary classification\n",
    "texts = [\n",
    "  'i left with my bouquet of red and yellow tulips under my arm feeling slightly more optimistic than when i arrived',\n",
    "  'i explain why i clung to a relationship with a boy who was in many ways immature and uncommitted despite the excitement i should have been feeling for getting accepted into the masters program at the university of virginia',\n",
    "  'i like to have the same breathless feeling as a reader eager to see what will happen next',\n",
    "  'i jest i feel grumpy tired and pre menstrual which i probably am but then again its only been a week and im about as fit as a walrus on vacation for the summer',\n",
    "  'i don t feel particularly agitated',\n",
    "  'i feel beautifully emotional knowing that these women of whom i knew just a handful were holding me and my baba on our journey',\n",
    "  'i pay attention it deepens into a feeling of being invaded and helpless',\n",
    "  'i just feel extremely comfortable with the group of people that i dont even need to hide myself',\n",
    "  'i find myself in the odd position of feeling supportive of',\n",
    "  'i was feeling as heartbroken as im sure katniss was',\n",
    "  'i feel a little mellow today',\n",
    "  'i feel like my only role now would be to tear your sails with my pessimism and discontent',\n",
    "  'i feel just bcoz a fight we get mad to each other n u wanna make a publicity n let the world knows about our fight',\n",
    "  'i feel like reds and purples are just so rich and kind of perfect']\n",
    "\n",
    "# Positive Sentiment = 1\n",
    "# Negative Sentiment = 0\n",
    "ground_truth = [ 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1]\n",
    "\n",
    "# Sample prediction\n",
    "predicted = [ 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "225eb2ad-9f51-42df-9e9e-c8a10a70c2ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_tp_fp_fn(ground_truth_list: list, predicted_list: list, positive_class) -> tuple:\n",
    "    true_positives = 0\n",
    "    false_positives = 0\n",
    "    false_negatives = 0\n",
    "    \n",
    "    for i in range(len(ground_truth_list)):\n",
    "        if ground_truth_list[i] == positive_class:\n",
    "            if predicted_list[i] == positive_class:\n",
    "                true_positives += 1\n",
    "            else:\n",
    "                false_negatives += 1\n",
    "        elif predicted_list[i] == positive_class:\n",
    "            false_positives += 1\n",
    "\n",
    "    return true_positives, false_positives, false_negatives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0eecfbd6-bed8-4b05-9b1d-85f32ab372b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True Positives: 5\n",
      "False Positives: 3\n",
      "False Negatives: 2\n"
     ]
    }
   ],
   "source": [
    "# Sample results\n",
    "positive_class = 1\n",
    "\n",
    "true_positives, false_positives, false_negatives = count_tp_fp_fn(ground_truth, predicted, positive_class)\n",
    "\n",
    "print(f\"True Positives: {true_positives}\")\n",
    "print(f\"False Positives: {false_positives}\")\n",
    "print(f\"False Negatives: {false_negatives}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ed1f92b-e5ad-4021-ac6a-24959431bc80",
   "metadata": {},
   "source": [
    "### F1 Score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7cd1a2b-6c90-44f7-8c1c-143ece73e29e",
   "metadata": {},
   "source": [
    "$precision = \\frac{TP}{TP + FP}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3c251ac7-9fe3-4565-bdc0-b00392cfa440",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision: 0.625\n"
     ]
    }
   ],
   "source": [
    "precision = true_positives / (true_positives + false_positives)\n",
    "print(f\"Precision: {precision:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5493cda6-9b93-471b-9a5d-d9645353bf1a",
   "metadata": {},
   "source": [
    "$recall = \\frac{TP}{TP+FN}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a856af08-4862-4b26-98e8-49846cde1b96",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recall: 0.714\n"
     ]
    }
   ],
   "source": [
    "recall = true_positives / (true_positives + false_negatives)\n",
    "print(f\"Recall: {recall:.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "952f724c-b191-4bd6-b337-2f1803f9e041",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision: 0.625\n",
      "Recall: 0.714\n"
     ]
    }
   ],
   "source": [
    "print(f\"Precision: {precision:.3f}\")\n",
    "print(f\"Recall: {recall:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ead8a4d-e908-455f-94d8-c16c74c6ab36",
   "metadata": {},
   "source": [
    "First Method: using precision and recall\n",
    "\n",
    "$F_1 = \\cfrac{2}{\\cfrac{1}{precision}+\\cfrac{1}{recall}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c727da5b-38f8-4143-82af-b8d6031e72c2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score calculated using precision and recall: 0.667\n"
     ]
    }
   ],
   "source": [
    "f1_score_a = 2 / ((1 / precision) + (1 / recall))\n",
    "print(f\"F1 Score calculated using precision and recall: {f1_score_a:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e73e575-8d16-44a8-ba20-a551be09453b",
   "metadata": {},
   "source": [
    "Second method using TP, FP and FN\n",
    "\n",
    "$F_1 = \\cfrac{TP}{TP + \\cfrac{FP+FN}{2}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "43ca7e2b-f98a-4a51-a37b-aa545555d164",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score calculated using TP FP and FN: 0.667\n"
     ]
    }
   ],
   "source": [
    "f1_score_b = true_positives / (true_positives + (false_positives + false_negatives) / 2)\n",
    "print(f\"F1 Score calculated using TP FP and FN: {f1_score_b:.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e96df1d5-afe8-43c1-a0f4-a631c23604bb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The two f1 scores are equal? True\n",
      "The two f1 scores are close up to 15 decimal places? True\n",
      "0.6666666666666666\n",
      "0.6666666666666666\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "print(f\"The two f1 scores are equal? {f1_score_a == f1_score_b}\")\n",
    "print(f\"The two f1 scores are close up to 15 decimal places? {math.isclose(f1_score_a, f1_score_b, abs_tol=0.0000000000000001)}\")\n",
    "print(f1_score_a)\n",
    "print(f1_score_b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "651f614c-0dac-468d-a846-f088eb1c1f5e",
   "metadata": {},
   "source": [
    "## Multiclass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43bab9d2-2ad8-416e-b5ed-7135eea182c0",
   "metadata": {},
   "source": [
    "- Dataset used for this sample\n",
    "<cite>\n",
    "  <a href=\"https://www.aclweb.org/anthology/D18-1404\">CARER: Contextualized Affect Representations for Emotion Recognition</a> by Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687-3697, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.\n",
    "</cite>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a5431815-740c-44d5-bcfc-b35e918ffbcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from https://github.com/dair-ai/emotion_dataset\n",
    "multi_class_texts = ['im feeling rather rotten so im not very ambitious right now',\n",
    "  'im updating my blog because i feel shitty',\n",
    "  'i never make her separate from me because i don t ever want her to feel like i m ashamed with her',\n",
    "  'i left with my bouquet of red and yellow tulips under my arm feeling slightly more optimistic than when i arrived',\n",
    "  'i was feeling a little vain when i did this one',\n",
    "  'i cant walk into a shop anywhere where i do not feel uncomfortable',\n",
    "  'i felt anger when at the end of a telephone call',\n",
    "  'i explain why i clung to a relationship with a boy who was in many ways immature and uncommitted despite the excitement i should have been feeling for getting accepted into the masters program at the university of virginia',\n",
    "  'i like to have the same breathless feeling as a reader eager to see what will happen next',\n",
    "  'i jest i feel grumpy tired and pre menstrual which i probably am but then again its only been a week and im about as fit as a walrus on vacation for the summer',\n",
    "  'i don t feel particularly agitated',\n",
    "  'i feel beautifully emotional knowing that these women of whom i knew just a handful were holding me and my baba on our journey',\n",
    "  'i pay attention it deepens into a feeling of being invaded and helpless',\n",
    "  'i just feel extremely comfortable with the group of people that i dont even need to hide myself',\n",
    "  'i find myself in the odd position of feeling supportive of',\n",
    "  'i was feeling as heartbroken as im sure katniss was',\n",
    "  'i feel a little mellow today',\n",
    "  'i feel like my only role now would be to tear your sails with my pessimism and discontent',\n",
    "  'i feel just bcoz a fight we get mad to each other n u wanna make a publicity n let the world knows about our fight',\n",
    "  'i feel like reds and purples are just so rich and kind of perfect']\n",
    "\n",
    "\n",
    "# 0: 'sadness'\n",
    "# 1: 'joy'\n",
    "# 2: 'love'\n",
    "# 3: 'anger'\n",
    "# 4: 'fear'\n",
    "# 5: 'surprise'\n",
    "ground_truth_multi = [0, 0, 0, 1, 0, 4, 3, 1, 1, 3, 4, 0, 4, 1, 2, 0, 1, 0, 3, 1]\n",
    "predicted_multi =    [0, 1, 2, 1, 2, 4, 3, 3, 1, 4, 4, 0, 4, 1, 2, 0, 1, 0, 3, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8756e867-9a1c-408b-bf0a-3e046ec2e510",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample Results\n",
    "n_class = 5\n",
    "multiclass_results_list = [count_tp_fp_fn(ground_truth_multi, predicted_multi, i) for i in range(n_class)]\n",
    "true_positives_list = [class_result[0] for class_result in multiclass_results_list]\n",
    "false_positives_list = [class_result[1] for class_result in multiclass_results_list]\n",
    "false_negatives_list = [class_result[2] for class_result in multiclass_results_list]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "02ee5401-6604-4a0d-b898-5f476e79f334",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 1, 2, 3]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "true_positives_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "eda6a4f9-a560-417d-91c0-6dbfd5d60c4e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 1, 1]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "false_positives_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "cd2fc9d6-15f4-4b53-94d1-ae557d8050d9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3, 1, 0, 1, 0]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "false_negatives_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6453f0b-8f2e-41c0-b89e-fe36ad0af724",
   "metadata": {},
   "source": [
    "### MacroF1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5843c29b-1ce6-4944-94f4-141140a9546d",
   "metadata": {},
   "source": [
    "$Macro F_1 = \\cfrac{\\sum_{i=1}^{n} F1 Score_i}{n}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b373bdb-85ab-4125-bfee-5c796880642b",
   "metadata": {},
   "source": [
    "Example for 2 classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9d65a81e-c061-4f1c-be6b-eeb703058369",
   "metadata": {},
   "outputs": [],
   "source": [
    "f1_score_0 = true_positives_list[0] / (true_positives_list[0] + (false_positives_list[0] + false_negatives_list[0]) / 2)\n",
    "f1_score_1 = true_positives_list[1] / (true_positives_list[1] + (false_positives_list[1] + false_negatives_list[1]) / 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7cae9db4-11a0-4e2c-86c7-c5dbb7c0140d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7803030303030303\n"
     ]
    }
   ],
   "source": [
    "macro_f1_score = (f1_score_0 + f1_score_1) / 2\n",
    "\n",
    "print(macro_f1_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34142807-1abb-416d-9165-34230860b8b1",
   "metadata": {},
   "source": [
    "Example for all classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c81cc1d6-959a-4797-81c4-f1682582218c",
   "metadata": {},
   "outputs": [],
   "source": [
    "f1_scores = [true_positives_list[i] / (true_positives_list[i] + (false_positives_list[i] + false_negatives_list[i]) / 2) for i in range(n_class)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9d3f7cc2-8e04-4550-bddc-6123ae72ede1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.7272727272727273, 0.8333333333333334, 0.5, 0.6666666666666666, 0.8571428571428571]\n"
     ]
    }
   ],
   "source": [
    "print(f1_scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b6747e14-b1de-4a9f-aa5c-b3d6b0522054",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7168831168831169\n"
     ]
    }
   ],
   "source": [
    "macro_f1_score = sum(f1_scores) / len(f1_scores)\n",
    "\n",
    "print(macro_f1_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ea9829e7-24c9-49df-9132-eec862b034b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from statistics import mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "d94a86cb-be49-4d29-8e0a-82f4888cd452",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7168831168831169"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean(f1_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c47b6c7-9c1c-4836-b1b2-0e97a9aead3e",
   "metadata": {},
   "source": [
    "### MicroF1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b64ce24d-44a2-47a3-aa43-8bad48b66b74",
   "metadata": {},
   "source": [
    "$Micro F_1 = \\cfrac{\\sum_{i=1}^{n} TP_i}{\\sum_{i=1}^{n} TP_i + \\cfrac{\\sum_{i=1}^{n} FP_i + \\sum_{i=1}^{n} FN_i}{2}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "3bbd4ac5-4280-41f0-9f97-61d995c50fd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro_f1_score = sum(true_positives_list) / (sum(true_positives_list) + ((sum(false_positives_list) + sum(false_negatives_list))/2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1ca1bd62-5cbe-423c-8e3e-728998a51ed2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.75\n"
     ]
    }
   ],
   "source": [
    "print(micro_f1_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "56a302c0-72d1-4dcf-87bd-59aede26ba5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "tp_sum = sum(true_positives_list)\n",
    "fp_sum = sum(false_positives_list)\n",
    "fn_sum = sum(false_negatives_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c1480d89-0540-4d2c-8ce2-0cdb06ad800f",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro_f1_score = tp_sum / (tp_sum + (fp_sum + fn_sum) / 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d4a32ad6-7fdf-4c41-b6f5-009cd4dc1b70",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.75\n"
     ]
    }
   ],
   "source": [
    "print(micro_f1_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c0a27d9-3b45-428a-b532-282d7a4914e7",
   "metadata": {},
   "source": [
    "## Scikit Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "0f073ec7-a44b-4bd4-8902-9785b00860b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: scikit-learn in ./venv/lib/python3.9/site-packages (1.3.0)\n",
      "Collecting scikit-learn\n",
      "  Downloading scikit_learn-1.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.9 MB)\n",
      "\u001b[K     |████████████████████████████████| 10.9 MB 4.2 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: joblib>=1.1.1 in ./venv/lib/python3.9/site-packages (from scikit-learn) (1.3.2)\n",
      "Requirement already satisfied: scipy>=1.5.0 in ./venv/lib/python3.9/site-packages (from scikit-learn) (1.11.2)\n",
      "Requirement already satisfied: numpy<2.0,>=1.17.3 in ./venv/lib/python3.9/site-packages (from scikit-learn) (1.25.2)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in ./venv/lib/python3.9/site-packages (from scikit-learn) (3.2.0)\n",
      "Installing collected packages: scikit-learn\n",
      "  Attempting uninstall: scikit-learn\n",
      "    Found existing installation: scikit-learn 1.3.0\n",
      "    Uninstalling scikit-learn-1.3.0:\n",
      "      Successfully uninstalled scikit-learn-1.3.0\n",
      "Successfully installed scikit-learn-1.3.1\n"
     ]
    }
   ],
   "source": [
    "!pip install -U scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "995cf880-5922-4685-8400-bb0348e1b768",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import f1_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "15e59402-49e1-4e43-aecb-26f56e07617f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.72727273, 0.83333333, 0.5       , 0.66666667, 0.85714286])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Per class\n",
    "f1_score(ground_truth_multi, predicted_multi, average=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "45bef144-e45d-4b15-b858-8f2d33c4d341",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7168831168831169"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Macro\n",
    "f1_score(ground_truth_multi, predicted_multi, average='macro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "69d16f90-e438-4026-9c63-202bd45291ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.75"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Micro\n",
    "f1_score(ground_truth_multi, predicted_multi, average='micro')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
